Towards Adaptive Training of Agent-based Sparring Partners for Fighter Pilots

摘要

A key requirement for the current generation of artificial decision-makers isthat they should adapt well to changes in unexpected situations. This paperaddresses the situation in which an AI for aerial dog fighting, with tunableparameters that govern its behavior, must optimize behavior with respect to anobjective function that is evaluated and learned through simulations. Bayesianoptimization with a Gaussian Process surrogate is used as the method forinvestigating the objective function. One key benefit is that duringoptimization, the Gaussian Process learns a global estimate of the trueobjective function, with predicted outcomes and a statistical measure ofconfidence in areas that haven't been investigated yet. Having a model of theobjective function is important for being able to understand possible outcomesin the decision space; for example this is crucial for training and providingfeedback to human pilots. However, standard Bayesian optimization does notperform consistently or provide an accurate Gaussian Process surrogate functionfor highly volatile objective functions. We treat these problems by introducinga novel sampling technique called Hybrid Repeat/Multi-point Sampling. Thistechnique gives the AI ability to learn optimum behaviors in a highly uncertainenvironment. More importantly, it not only improves the reliability of theoptimization, but also creates a better model of the entire objective surface.With this improved model the agent is equipped to more accurately/efficientlypredict performance in unexplored scenarios.